SPECIFICATION 



TO ALL WHOM IT MAY CONCERN : 

BE IT KNOWN THAT WE, Shigeru Nishio, a 
citizen of Japan residing at Kawasaki, Japan, Hiroshi Daiku, 
a citizen of Japan residing at Kawasaki, Japan and Asao 
Kokubo, a citizen of Japan residing at Kawasaki, Japan have 
invented certain new and useful improvements in 



CIRCUIT AND METHOD FOR CORRECTION OF DEFECT PIXEL 



Of which the following is a specification:- 



TITLE OF THE INVENTION 

CIRCUIT AND METHOD FOR CORRECTION OF 
DEFECT PIXEL 



CROSS-REFERENCE TO RELATED APPLICATIONS 

The present application is based upon and 
claims the benefit of priority from the prior 
Japanese Patent Application No. 2002-316077 filed on 
October 30, 2002, with the Japanese Patent Office, 
the entire contents of which are incorporated herein 
by reference. 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention generally relates to 
the processing of image signals supplied from a CMOS 
image sensor for image pickup, and particularly 
relates to the correction of defect pixels contained 
in the image signals from a CMOS image sensor. 

2 . Description of the Related Art 

A CMOS image sensor includes a reception 
section in which a plurality of photo-diodes are 
arranged in a matrix array. These photo-diodes 
correspond to individual pixels for the imaging 
purpose. Incident light is subjected to 
photoelectric conversion on a pixel-by-pixel basis, 
and the electric charge obtained by the 
photoelectric conversion is accumulated in an 
electric charge storage section for reading of data. 
A large number of pixels inevitably include 
defective pixels that do not operate properly, which 
appear as fixed dark dots (black dots) or bright 
dots (white dots) 

Fig.l is a block diagram showing the 
construction of a unit for defect correction of a 
CMOS image sensor. 

The unit of Fig.l includes a defect 
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image sensor, determining, based on the comparison, 
whether the pixel of interest is defective, and 
correcting the value of the pixel of interest by 
using values of surrounding pixels in response to 
5 the determination that the pixel of interest is 
defective . 

The method of correcting a defect pixel as 
described above can cope with any numbers of defects 
without a particular limitation, and can also cope 
10 with such defects as occurring through deterioration 
with time. 

Other objects and further features of the 
present invention will be apparent from the 
following detailed description when read in 
15 conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig.l is a block diagram showing the 
construction of a unit for defect correction of a 
20 CMOS image sensor; 

Fig. 2 is a block diagram showing the 
construction of a unit for defect correction of a 
CMOS image sensor according to the present 
invention ; 

25 Fig. 3 is a block diagram showing the 

construction of a defect correcting unit and a check 
circuit shown in Fig. 2; 

Fig. 4 is a flowchart showing a method of 
correcting defect pixels that is performed by the 
30 construction shown in Fig. 3; 

Figs.5A and 5B are illustrative drawings 
for explaining pixels subjected to comparison; 

Fig. 6 is an illustrative drawing for 
explaining a comparison check with a dead zone; 
35 Fig. 7 is a block diagram showing another 

configuration for defect correction of a CMOS image 
sensor according to the present invention; and 
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As shown in Fig. 3, the defect correcting 
unit 21 includes a corrected-value generating unit 
31 and a switch unit 32. The check circuit 22 
includes a horizontal-direction pixel comparison 
5 unit 35, a vertical-direction pixel comparison unit 
36, and a defect checking unit 37. 

Fig. 4 is a flowchart showing a method of 
correcting defect pixels that is performed by the 
construction shown in Fig. 3. In the following, the 
10 correction of defect pixels according to the 

invention will be described with reference to Fig. 3 
and Fig . 4 . 

At step ST1 of Fig. 4, pixel comparison is 
made in the horizontal direction. Specifically, 

15 data of each pixel is successively supplied from the 
image sensor to the defect correcting unit 21. The 
data of each pixel is in turn supplied from the 
defect correcting unit 21 to the horizontal- 
direction pixel comparison unit 35 of the check 

20 circuit 22, by which pixel comparison is made in the 
horizontal direction . 

Figs.5A and 5B are illustrative drawings 
for explaining pixels subjected to comparison. 
Fig.5A shows a RGB Bayer array. In the typical 

25 output of an image sensor, each pixel does not have 
the data indicative of three R, G, and B colors. 
Rather, each pixel represents one pre-assigned color 
among the RGB colors as shown in Fig.5A. The RGB 
array shown in Fig.5A is commonly referred to as the 

30 RGB Bayer array. Based on the RGB Bayer array, the 
RGB conversion unit 23 performs a process for 
assigning the three RGB colors to each pixel. 

Fig.5B is an illustrative drawing showing 
the positional relationship between neighboring 

35 pixels having the same color in the RGB Bayer array. 
As can be seen from Fig.5A, pixels having the same 
color in the RGB Bayer array are located at every 
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other pixel position in the vertical direction as 
well as in the horizontal direction. When a 
particular color (e.g., R) is considered, with a 
pixel of interest p[x, y] noted as having this color, 
5 the four neighboring pixels having the same color 
are p[x, y-2], p[x-2, y] , p[x+2, y] , and p[x, y+2]. 

At step ST1 of Fig. 4, it is p[x-2, y] and p[x+2, 
y] that are subjected to horizontal pixel comparison 
in respect of the pixel of interest p[x, y] . 

10 Specifically, a check is made as to whether the 
following condition is satisfied. 

(p[x-2, y] < p[x, y] < p[x+2, y]) or (p[x-2, y] > p[x, y] > p[x+2, y]) (1) 
If the condition (1) is satisfied, the pixel of 
interest is ascertained to be a normal pixel . In 

15 such a case, the procedure comes to an end without 
performing a correction process in the flowchart of 
Fig. 4. Accordingly, the value of the pixel of 
interest stays the same as originally provided in 
the input signals. 

20 The operation described above will now be 

fit into the construction shown in Fig. 3. If the 
horizontal-direction pixel comparison unit 35 finds 
the pixel of interest to be a normal pixel, the 
vertical-direction pixel comparison unit 36 and the 

25 defect checking unit 37 do not operate, for example. 
As a result, the switch unit 32 selects the input 
image signals from the image sensor as a default 
selection, and passes the selected signals to the 
output . 

30 If the condition (1) is not satisfied, at 

step ST2 of Fig. 4, a comparison is made with the 
dead zone in the horizontal direction. Specifically, 
the horizontal-direction pixel comparison unit 35 of 
the check circuit 22 performs a comparison check 

35 based on the data of each supplied pixel as shown in 
the following. 

p[x, y] < Wlimit x (p[x-2, y] + p[x+2, y])/2 + Woffset (2) 
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p[x, y] > Blimit x (p[x-2, y] + p[x+2, y])/2 - Boffset (3) 

Here, Wlimit and Woffset are constants that define 
the upper limit of the dead zone. For example, 
Wlimit may be around 1.2, and Woffset may be around 
5 20 to 30 in the gray scale of 256 levels. Blimit 
and Boffset are constants that define the lower 
limit of the dead zone. For example, Blimit may be 
around 0.8, and Boffset may be around 20 to 30 in 
the gray scale of 256 levels. Wlimit and Blimit may 

10 be the same value, or may be set to different values. 
Woffset and Boffset may be the same value, or may be 
set to different values. 

Fig. 6 is an illustrative drawing for 
explaining the comparison check with the dead zone. 

15 In Fig. 6, the horizontal axis represents 

the position N of the pixel of interest and 
positions N-2 and N+2 of the neighboring pixels. 
The vertical axis represents the pixel value (i.e., 
pixel code) of each pixel. In this example, the 

20 pixel of interest 41 has the largest pixel value, 
and the neighboring pixels 42 and 43 have pixel 
values as shown in Fig. 6. A range A is defined by 
the pixel value of the neighboring pixel 42 and the 
pixel value of the neighboring pixel 43. If the 

25 pixel value 41 falls within this range A, the 

condition (1) corresponding to the check of step STl 
is satisfied. A range C is defined by a dead-zone 
upper limit 44 (the value of the right-hand side of 
the inequality (2)) and a dead-zone lower limit 45 

30 (the value of the right-hand side of the inequality 
(3)) , which are obtained based on an average B of 
the pixel values of the neighboring pixels 42 and 43. 
If the pixel of interest 41 falls within the range C, 
the inequalities (2) and (3) corresponding to the 

35 check of step ST2 are satisfied. 

If both of the inequalities (2) and (3) 
are satisfied, the pixel of interest is ascertained 
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to be a normal pixel. In such a case, the procedure 
comes to an end without performing a correction 
process in the flowchart of Fig. 4. Accordingly, the 
value of the pixel of interest stays the same as 
5 originally provided in the input signals. 

This operation will now be fit into the 
construction of Fig. 3. If the horizontal-direction 
pixel comparison unit 35 finds the pixel of interest 
to be a normal pixel, the vertical-direction pixel 

10 comparison unit 36 and the defect checking unit 37 
do not operate, for example. As a result, the 
switch unit 32 selects the input image signals from 
the image sensor as a default selection, and passes 
the selected signals to the output. 

15 If the conditions (2) and (3) are not 

satisfied, at step ST3 of Fig. 4, pixel comparison is 
made in the vertical direction. Specifically, when 
data of each pixel is successively supplied from the 
image sensor to the defect correcting unit 21 in the 

20 construction of Fig. 3, the data of each pixel is in 
turn supplied from the defect correcting unit 21 to 
the vertical-direction pixel comparison unit 36 via 
the horizontal-direction pixel comparison unit 35. 
The vertical-direction pixel comparison unit 36 

25 performs pixel comparison in the vertical direction. 

Pixel values subjected to the vertical 
pixel comparison are p[x, y-2] and p[x, y + 2]. 
Specifically, a check is made as to whether the 
following condition is satisfied. 

3 0 (p[x, y-2] < p[x, y] < p[x, y+2]) or (p[x, y-2] > p[x, y] > p[x, y+2]) (4) 

If the condition (4) is satisfied, the pixel of 
interest is ascertained to be a normal pixel. In 
such a case, the procedure comes to an end without 
performing a correction process in the flowchart of 

35 Fig. 4. Accordingly, the value of the pixel of 

interest stays the same as originally provided in 
the input signals. 
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The operation described above will now be 
fit into the construction shown in Fig. 3. If the 
vertical-direction pixel comparison unit 36 finds 
the pixel of interest to be a normal pixel, the 
5 defect checking unit 37 does not operate, for 

example. As a result, the switch unit 32 selects 
the input image signals from the image sensor as a 
default selection, and passes the selected signals 
to the output. 

10 If the condition (4) is not satisfied, at 

step ST4 of Fig. 4, a comparison is made with the 
dead zone in the vertical direction. Specifically, 
the vertical-direction pixel comparison unit 36 of 
the check circuit 22 performs a comparison check 

15 based on the data of each supplied pixel as shown in 
the following. 

p[x, y] < Wlimit x (p[x, y-2] + p[x, y+2])/2 + Woffset (5) 
p[x, y] > Blimit x (p[x, y-2] + p[x, y+2])/2 - Boffset (6) 
Here, Wlimit and Woffset are constants that define 

20 the upper limit of the dead zone. For example, 

Wlimit may be around 1.2, and Woffset may be around 
20 to 30 in the gray scale of 256 levels. Blimit 
and Boffset are constants that define the lower 
limit of the dead zone. For example, Blimit may be 

25 around 0.8, and Boffset may be around 20 to 30 in 

the gray scale of 256 levels. Wlimit and Blimit may 
be the same value, or may be set to different values. 
Woffset and Boffset may be the same value, or may be 
set to different values. 

30 If both of the inequalities (5) and (6) 

are satisfied, the pixel of interest is ascertained 
to be a normal pixel. In such a case, the procedure 
comes to an end without performing a correction 
process in the flowchart of Fig. 4. Accordingly, the 

35 value of the pixel of interest stays the same as 
originally provided in the input signals. 

This operation will now be fit into the 
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construction of Fig. 3. If vertical-direction pixel 
comparison unit 36 finds the pixel of interest to be 
a normal pixel, the defect checking unit 37 does not 
operate, for example. As a result, the switch unit 
5 32 selects the input image signals from the image 
sensor as a default selection, and passes the 
selected signals to the output. 

If either the condition (5) or the 
condition (6) is not satisfied, a corrected value is 

10 generated at step ST5 of Fig. 4, and the correction 
process is performed on the pixel of interest. 
Specifically, the corrected-value generating unit 31 
of the defect correcting unit 21 carries out a 
process as follows, thereby obtaining a corrected 

15 value in respect of the pixel of interest that is 
defective. If the condition (5) is not specified, 
the pixel of interest is ascertained to be a bright 
(white) defect pixel, followed by this computation. 
p[x, y] = p[x, y] x Wweight 

2 0 + (p[x-2, y] + p[x+2, y] + p[x, y-2] + p[x, y+2])/4 x (1 -Wweight) (7) 

If the condition (6) is not specified, the pixel of 
interest is ascertained to be a dark (black) defect 
pixel, followed by this computation. 
p[x, y] = p[x, y] x Bweight 

2 5 + (p[x-2, y] + p[x+2, y] + p[x, y-2] + p[x, y+2])/4 x (1 -Bweight) (8) 

Here, Wweight and Bweightb are factors for adjusting 
relative weight of the pixel of interest and the 
four surrounding pixels when the pixel of interest 
is corrected. When these factors are close to 1, 

30 the original value of the pixel of interest is 
relatively respected in obtaining the corrected 
value. When these factors are close to 0, the 
original value of the pixel of interest is 
relatively ignored, and the values of the 

35 surrounding pixels are mainly used in obtaining the 
corrected value. Wweight and Bweight may be the 
same value, or may be set to different values. 
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If Wweight and Bweight are set to the same 
value, the equations (7) and (8) become identical, 
so that the cor rected-value generating unit 31 may 
perform the same process for generating a corrected 
5 value regardless of the determination of conditions. 
If Wweight and Bweight are set to different values, 
the corrected-value generating unit 31 performs 
different processes for generating a corrected value, 
depending on which one of the equations (7) and (8) 

10 is not satisfied. In this case, provision is made 

such that the corrected-value generating unit 31 can 
refer to the check determination made by the 
vertical-direction pixel comparison unit 36, as 
indicated by dotted lines in Fig. 3. 

15 in Fig. 3, the corrected-value generating 

unit 31 generates a corrected value, and, at the 
same time, the vertical-direction pixel comparison 
unit 36 supplies a signal to the defect checking 
unit 37 for indicating that one of the equations (5) 

20 and (6) is not satisfied. In response to this 

signal, the defect checking unit 37 ascertains that 
the pixel of interest is defective, thereby setting 
the switch unit 32 to select the output of the 
corrected-value generating unit 31. As a result, 

25 the corrected value generated by the corrected-value 
generating unit 31 is supplied to a next processing 
stage with respect to the pixel of interest. 

In this manner, the defect correction 
according to the invention compares the value of the 

30 pixel of interest with the values of surrounding 
pixels in both the horizontal direction and the 
vertical direction, thereby determining whether the 
value of the pixel of interest differs by more than 
a predetermined margin from the values of the 

35 surrounding pixels, i.e., determining whether the 
pixel of interest is sticking out from the 
surrounding pixels. According to the determination, 
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th e pixel of Interest Is ascertained to be a defect 
pixel if the pixel of interest is sticking out. In 
this case, the value of the pixel of interest is 
corrected according to the values of the surrounding 
5 pixels. Unlike the related-art configuration in 

which a defect position storage device such as a ROM 
is employed, the invention can cope with any number 
of defects without a particular limitation, and can 
also cope with such defects as occurring through 

10 deterioration with time. 

In the defect correction of the invention, 
the horizontal check and the vertical check are 
independently performed, and the pixel of interest 
is ascertained to be defective only if both of the 

15 checks determine that the pixel of interest is a 
defect pixel. Even if a prominent vertical or 
horizontal line is present with pixel values 
differing significantly from surrounding pixel 
values, either the horizontal check or the vertical 

20 check produces a determination indicative of the 

case of normal pixels, resulting in no correction of 
pixel values. In this manner, the invention can 
ascertain that prominent horizontal or vertical 
lines are normal image data, thereby avoiding 

25 erroneously erasing these lines. 

Fig. 7 is a block diagram showing another 
configuration for defect correction of a CMOS image 
sensor according to the present invention. In Fig. 7, 
the same elements as those of Fig. 3 are referred to 

30 by the same numerals, and a description thereof will 
be omitted unless it is necessary. 

In the configuration of Fig. 7, horizontal 
pixel comparison by the horizontal-direction pixel 
comparison unit 35 and vertical pixel comparison by 

35 the vertical-direction pixel comparison unit 36 are 
performed in parallel. Namely, the data of each 
pixel that is successively supplied from the image 
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sensor to the defect correcting unit 21 is In turn 
supplied to the horizontal-direction pixel 
comparison unit 35 as well as to the vertical- 
direction pixel comparison unit 36 in the check 
5 circuit 22A. The horizontal-direction pixel 

comparison unit 35 and the vertical-direction pixel 
comparison unit 36 performs respective check 
operations simultaneously, and supply respective 
determinations to the defect checking unit 37A. The 

10 defect checking unit 37A ascertains the pixel of 
interest to be defective if both the horizontal- 
direction pixel comparison unit 35 and the vertical- 
direction pixel comparison unit 36 ascertain that 
the pixel of interest is a defect pixel . In such a 

15 case, the defect checking unit 37A sets the switch 
unit 32 to select the output of the cor rected-value 
generating unit 31. 

The configuration as described above 
produces the same defect determination as does the 

20 configuration of Fig. 3. 

In the configuration of Fig. 7, the 
horizontal-direction pixel comparison unit 35 and 
the vertical-direction pixel comparison unit 36 make 
respective comparisons concurrently . Alternatively , 

25 comparison by the horizontal-direction pixel 

comparison unit 35 may be made after the vertical- 
direction pixel comparison unit 36 makes a 
comparison. Namely, the horizontal-direction pixel 
comparison unit 35 and the vertical-direction pixel 

30 comparison unit 36 may exchange positions in he 

configuration of Fig. 3, thereby reversing the order 
in which horizontal comparison and vertical 
comparison are made. Such configuration produces 
the same defect determination as does the 

35 configuration of Fig. 3. 

Fig. 8 is a block diagram showing the 
construction of an image processor to which defect 
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correctlon of the Invention Is applied. 

The image processor of Fig. 8 includes a 
defect correcting unit 51, a RGB conversion unit 52, 
a white balancing unit 53 , a contour enhancing unit 
5 54, a gamma correction unit 55, and a format 
conversion unit 56. 

The defect correcting unit 51 receives 
image signals from an image sensor such as a CCD 
(charge coupled device) , and carries out defect 

10 correction of the invention. The image signals 

having undergone the defective pixel correction are 
supplied from the defect correcting unit 51 to the 
RGB conversion unit 52 . The RGB conversion unit 52 
outputs image signals by obtaining color data of 

15 each pixel based on color information of the RGB 

Bayer array. The image signals output from the RGB 
conversion unit 52 are then subjected to white 
balancing by the white balancing unit 53. After 
this, the contour enhancing unit 54 carries out a 

20 process for enhancing contours. The gamma 

correction unit 55 applies a process for correcting 
gamma characteristics of the output apparatus . 
Finally, the format conversion unit 56 converts the 
image signals into a format acceptable to a 

25 subsequent processing stage, and the converted image 
signals are output. 

Further, the present invention is not 
limited to these embodiments, but various variations 
and modifications may be made without departing from 

30 the scope of the present invention. 



